Volumetric data sets routinely captured in imaging processes within many different technical fields are rapidly increasing in size due to improved imaging modalities. The large data sets imply new analysis possibilities, but do also cause severe performance limitations in visualization pipelines due to the large amount of data to be processed. A typical visualization pipeline comprises the steps of data capture, compression, storage, decompression and rendering. Furthermore, the step of rendering typically comprises the use of a transfer function, which i.a. describes the important range. When a transfer function is applied, large subsets of data give little or no contribution to the rendered image.
One field in which imaging is a very important tool is medicine. Medical imaging may be used for creation and analysis of medical images of an interior region of a body of a patient. Doctors and other medical personnel may use medical imaging in different stages of medical procedures such as diagnosis of, for example, injuries, diseases and abnormal conditions, surgery planning, treatment and postoperative evaluation. The medical data may, for example, be acquired utilizing computerized tomography (CT), nuclear magnetic resonance (NMR), magnetic resonance imaging (MRI), ultrasound, X-ray angiography or positron emission tomography (PET).
The size of standard volumetric data sets in medical imaging is rapidly increasing due to that newly developed medical imaging modalities provide for, for example, improved geometric resolution and decreased examination time, i.e. the imaging modalities are more efficient such that the time a specific examination procedure takes is decreased whereby more data may be captured during the limited time a patient is subject to examination.
Since increasingly precise information may be gathered in medical imaging, improved diagnostic procedures are possible and new types of examinations may be developed. For example, invasive methods may thereby be replaced with non-invasive methods to minimize patient risk and discomfort. However, as mentioned above, large data sets cause severe performance limitations in visualization pipelines due to the large amount of data to be processed.
In medical imaging, a medical transfer function is typically used in the step of rendering. The medical transfer function describes the diagnostically important range and sets color and opacity for tissue types. Often, more than 50% of the voxels do not contribute to the rendered image and, for example, a typical medical transfer function for CT volumes makes tissue with attenuation lower than fat completely transparent.
Potentially, the most important visualization method for medical diagnostic work on medical volumetric image data is “Direct Volume Rendering” (DVR). It is, however, a difficult task to introduce DVR into the diagnostic workflow (Andriole, 2003). Technical limitations in terms of memory and bandwidth pose challenges for the visualization pipeline, making interactive frame rates hard to reach and maintain.
Direct Volume Rendering techniques (Kaufman, 1991) have been the focus of vast research efforts in recent years. Many researchers have attempted to reduce the amount of data to be processed in the DVR pipeline while maintaining rendering quality.
Westermann, 1994, presents a multi-resolution framework for DVR where ray-casting rendering with adaptive sampling frequency is performed directly on the wavelet transformed data. Schneider et al., 2003, propose a compression and rendering scheme for DVR based on vector quantization. An advantage of this approach is the ability to both decompress and render on the graphics hardware. Furthermore, Guthe et al., 2002, achieve a multi-resolution representation through a blocked wavelet compression scheme. A level-of-detail (LOD) selection occurs in the decompression stage, whereby block resolution partly is prioritised according to the reconstruction error of different LODs.
Furthermore, some of the known methods for reduction of the amount of data to be processed in the DVR pipeline utilize the knowledge encoded in the transfer function in order to guide a selection of a level-of-detail (LOD).
Bajaj et al., 2001, explore the use of voxel visualization importance in the compression process by utilizing the knowledge encoded in the transfer function in the compression step. Voxel weights are defined, e.g. for DVR on the basis of transfer functions. Each wavelet coefficient is then modulated by the maximum weight in the voxel set that contributed to the coefficient. This aims to give the coefficients with most visual importance the largest magnitude. Furthermore, it is shown that application of a threshold to weighted coefficients yields higher quality than using unweighted ones. However, a drawback with this scheme is that the important visualization features need to be known at compression time. A further limitation for the resulting image quality is the use of the simple Haar wavelet. Introduction of a more advanced wavelet would make the weighting less precise, since each coefficient will depend on many more voxels if the wavelet filter support data increases. Still a further drawback of this scheme is that this scheme introduces lossy compression.
Sohn et al., 2002, suggest the use of volumetric features to guide compression, which in this case is applied to time-varying volumes. The features are defined in terms of iso-surface values or intensity ranges. Even though transfer functions are not explicitly used, the volumetric features represent the same type of visualization importance. The data is first passed through a block-based Haar wavelet compression stage. Blocks that have little or no contribution to the selected features are discarded. The wavelet coefficients can also be thresholded depending on their contribution to the features. However, a major limitation of this scheme too is that features must be selected before compression occurs.
The work by Li et al., 2002, aims to achieve constant frame rates for volume rendering. The volume is divided into subvolumes of varying size, where coherent regions result in larger subvolumes. A multi-resolution pyramid for each subvolume is created by straight-forward averaging. Rendering time budgets are allocated to subvolumes according to an importance value which can be controlled, among other factors, by the maximum opacity in the subvolume. The full transfer function is not used, only the opacity component. This work does not explicitly require features to be selected at compression time, but on the other hand there is no description of how to handle feature importance changes, e.g. a change of transfer function. Furthermore, it is not shown how data reduction can be utilized outside the rendering stage, e.g. to lessen the impact of a low network bandwidth.
None of the above methods is capable of being incorporated with arbitrary existing visualization pipelines using multi-resolved data, for instance pipelines based on the JPEG2000 standard.
Thus, there is need for a simple way of achieving the ability to reduce the amount of data to be processed in a visualization pipeline while maintaining rendering quality and without requiring knowledge of rendering parameters or the transfer function in the step of compression, which would hinder transfer function changes during rendering. Furthermore, there is a need for such a method that do not introduce lossy compression and that may be incorporated with arbitrary existing visualization pipelines using multi-resolved data.